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(54) Apparatus and method for demultiplexing multiplexed data 



(57) Disclosed is an apparatus for demultiplexing 
multiplexed data. This apparatus comprises a demulti- 
plexer for receiving a multiplexed packet stream in 
which a plurality of object data are multiplexed, and 
demultiplexing the multiplexed data with reference to 
packet identification numbers added to the respective 
packets; a decoder for decoding M pieces of object data 
separated from the multiplexed data, and outputting M 
pieces of decoded object data; a memory for storing N 
pieces of information relating to a program and sepa- 
rated, by the demultiplexer; a memory for storing object 
composition information separated by the demulti- 
plexer; a memory for storing information relating to the 
object data and separated by the demultiplexer; a com- 
positor for compositing the M pieces of decoded object 
data; and an information analyzer for analyzing the 
packet identification numbers from the N pieces of infor- 
mation relating to the program, the object composition 
information, or the information relating to the object 
data. Therefore, when a multiplexed packet stream in 
which MPEG4 data are multiplexed by MPEG2 is 
demultiplexed, the MPEG4 data can be separated and 
decoded with high reliablity. 
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Description 

FIELD OF THE INVENTION 

The present invention relates to an apparatus and a 
method for demultiplexing a packet stream in which var- 
ious kinds of data are multiplexed, and a data storage 
medium containing a program for implementing the 
demultiplexing process by software. More specifically, 
the invention relates to demultiplexing of a packet 
stream in which various kinds of data are multiplexed by 
MPEG2 which is an international standard of image 
compressive coding, or a packet stream in which a plu- 
rality of object data are multiplexed by MPEG4 which is 
currently being standardized. 

BACKGROUND OF THE INVENTION 

In recent years, MPEG4 has been discussed as a 
method for transmitting digital data of video, audio, and 
control information in packets (reference literatures: 
ISO/IEC JTC1/SC29/WG11 N1692 MPEG4-Systems 
Working Draft 4.0, ISO/IEC JTC1/SC29/WG1 1 N1693 
MPEG4-Systems Verification Model 4.0). In MPEG4, 
digital data of compressively coded video and audio, 
object composition information describing hierarchy of 
objects, and object descriptors describing coding 
schemes of the objects are subjected to the MPEG4 
packetization using the Access Unit Layer, and these 
MPEG4 packets are further packetized by another mul- 
tiplexing protocol such as MPEG2 (reference literature: 
ISO/IEC JTC1/SC29/WG1 1 N801 "ISO/IEC 13818-1). 
On the other hand, in MPEG2, video and audio data 
compressively coded by MPEG2 are packetized in a for- 
mat called "PES (Packetized Elementary Stream) 
Packet", and the PES packet is divided into fixed-length 
packets called "TS (Transport Stream) packets". 

In MPEG4, standardization for which is currently in 
progress, since the MPEG4 packetization using the 
Access Unit Layer is indispensable, video and audio 
data compressively coded by MPEG2 are also sub- 
jected to the MPEG4 packetization. Therefore, when an 
MPEG2 data receiver for receiving satellite broadcast- 
ing or CATV broadcasting, which have been spread in 
recent years, receives an MPEG4 data stream in which 
video and audio data compressively coded by MPEG2 
are MPEG4-packet*zed and multiplexed, the receiver 
cannot separate and decode the video and audio data 
compressed by MPEG2. 

Furthermore, a broadcasting receiving terminal 
receives object composition information describing hier- 
archy of objects, and object descriptors describing cod- 
ing schemes of the objects, several times, even though 
the contents of these data are not changed. This repeti- 
tion is necessary because there is a possibility that 
receiving is started at an arbitrary timing in the broad- 
casting receiving terminal. The object composition infor- 
mation and the object descriptors are also subjected to 



MPEG4 packetization using the Access Unit Layer as 
well as video and audio data. However, when the receiv- 
ing terminal receives such data of the same contents 
repeatedly, the burden on the receiving terminal may be 
5 increased. 

Furthermore, MPEG4 does not provide a method 
for specifying object composition information, object 
descriptors, and video and audio data compressively 
coded by MPEG4, from multiplexed data. 

10 

SUMMARY OF THF INVENTION 

It is an object of the present invention to provide an 
apparatus and a method for demultiplexing multiplexed 
15 data, that enable reliable separation and decoding of 
MPEG4 data even through the MPEG4 data is multi- 
plexed by MPEG2. 

Other objects and advantages of the invention will 
become apparent from the detailed description that fol- 
20 lows. The detailed description and specific embodi- 
ments described are provided only for illustration since 
various additions and modifications within the spirit and 
scope of the invention will be apparent to those skilled in 
the art from the detailed description. 
25 According to a first aspect of the present invention, 
there is provided a demultiplexing apparatus compris- 
ing: demultiplexing means for receiving a multiplexed 
packet stream in which a plurality of object data are mul- 
tiplexed, and demultiplexing the multiplexed data with 
30 reference to packet identification numbers added to the 
respective packets; decoding means for decoding M 
pieces of object data separated from the multiplexed 
data, and outputting M pieces of decoded object data; a 
memory for storing N pieces of information relating to a 
35 program and separated by the demultiplexing means; a 
memory for storing object composition information sep- 
arated by the demultiplexing means; a memory for stor- 
ing information relating to the object data and separated 
by the demultiplexing means; composition means for 
40 compositing the M pieces of decoded object data; and 
information analysis means for analyzing the packet 
identification numbers from the N pieces of information 
relating to the program, the object composition informa- 
tion, or the information relating to the object data. 
ts According to a second aspect of the present inven : 
tion, in the above-described demultiplexing apparatus, 
the information analysis means analyzes the N pieces 
of information relating to the program, obtains a packet 
identification number PID_syn of a packet containing 
'0 the object composition information, and outputs the 
packet identification number PID_syn to the demulti- 
plexing means; and the demultiplexing means sepa- 
rates the packet containing the object composition 
information from the input multiplexed packet stream 
5 with reference to the packet identification number 
PID_syn, and outputs this packet. 

According to a third aspect of the present invention, 
in the above-described demultiplexing apparatus, the 
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information analysis means analyzes the N pieces of 
information relating to the program, obtains a packet 
identification number PID_od of a packet containing the 
information relating to the object data, and outputs the 
packet identification number PID_od to the demultiplex- 
ing means; and the demultiplexing means separates the 
packet containing the information relating to the object 
data from the input multiplexed packet stream with refer- 
ence to the packet identification number PID__od, and 
outputs this packet. 

According to a fourth aspect of the present inven- 
tion, in the above-described demultiplexing apparatus, 
the information analysis means analyzes the object 
composition information, and outputs information indi- 
cating the hierarchical relationship of the object data 
toward the composition means; and the composition 
means composites the M pieces of decoded object data 
with reference to the hierarchical relationship. 

According to a fifth aspect of the present invention, 
in the above-described demultiplexing apparatus, the 
information analysis means analyzes the information 
relating to the object data, judges a code ALpredef ined 
for each of the object data, which code indicates 
whether packetization according to the international 
standard MPEG4 exists or not, and outputs the code 
Al_predefined toward the demultiplexing means; and 
the demultiplexing means performs separation of the 
MPEG4 packetization for each object data with refer- 
ence to the code Al _predef ined. 

According to a sixth aspect of the present invention, 
in the above-described demultiplexing apparatus, the 
information analysis means analyzes the information 
relating to the object data, judges a code Al_predef ined 
for each of the object data, which code indicates 
whether packetization according to the international 
standard MPEG4 exists or not, and outputs the code 
ALpredefined toward the decoding means; and the 
decoding means performs separation of the MPEG4 
packetization with reference to the code ALpredefined. 

According to a seventh aspect of the present inven- 
tion, in the above-described demultiplexing apparatus, 
the information analysis means analyzes the informa- 
tion relating to the object data, judges a code AI_config 
for each of the object data, which code indicates setting 
information of a packet header according to the interna- 
tional standard MPEG4, and outputs the code AI„config 
toward the demultiplexing means; and the demultiplex- 
ing means performs separation of the MPEG4 packeti- 
zation for each object data with reference to the code 
AI__config. 

According to an eighth aspect of the present inven- 
tion, in the above-described demultiplexing apparatus, 
the information analysis means analyzes the informa- 
tion relating to the object data, judges a code AI_config 
for each of the object data, which code indicates setting 
information of a packet header according to the interna- 
tional standard MPEG4, and outputs the code AI_config 
toward the decoding means; and the decoding means 



performs separation of the MPEG4 packetization with 
reference to the code AI_conf ig. 

According to a ninth aspect of the present inven- 
tion, in the above-described demultiplexing apparatus, 
5 the information analysis means analyzes the N pieces 
of information relating to the program, obtains a packet 
identification number PID_d of a packet containing the 
object data and a packet identification number FMC__d 
according to the international standard MPEG4, and 
10 outputs these identification numbers toward the demul- 
tiplexing means; and the demultiplexing means sepa- 
rates the packet containing the object data from the 
input multiplexed packet stream with reference to the 
identification numbers PID_d and FMC_d, and outputs 
15 the packet. 

According to a tenth aspect of the present inven- 
tion, in the above-described demultiplexing apparatus, 
the information analysis means obtains a code for 
deciding whether the packet identification number 
20 FMC_d according to the international standard MPEG4 
is added or not, by analyzing the N pieces of information 
relating to the program. 

According to an eleventh aspect of the present 
invention, in the above-described demultiplexing appa- 
25 ratus, the information analysis means receives the 
object composition information which is divided into at 
least one section, and integrates these sections with 
reference to version numbers and section numbers 
which are added at the heads of the respective sec- 
30 tions, thereby obtaining the object composition informa- 
tion. 

According to a twelfth aspect of the present inven- 
tion, in the above-described demultiplexing apparatus, 
the information analysis means receives the information 

35 relating to the object data, which information is divided 
into at least one section, and integrates these sections 
with reference to version numbers and section numbers 
which are added at the heads of the respective sec- 
tions, thereby obtaining the information relating to the 

40 object data. 

According to a thirteenth aspect of the present 
invention, in the above-described demultiplexing appa- 
ratus, the information analysis means receives the 
object composition information which is divided into at 

45 least one section, decides whether data of each section 
has a data error or not according to an error detection 
code added to the end of each section, and discards the 
section having a data error. 

According to a fourteenth aspect of the present 

so invention, in the above-described demultiplexing appa- 
ratus, the information analysis means receives the infor- 
mation relating to the object data, which is divided into 
at least one section, decides whether data of each sec- 
tion has a data error or not according to an error detec- 

55 tion code added to the end of each section, and 
discards the section having a data error. 

According to a fifteenth aspect of the present inven- 
tion, there is provided a demultiplexing method compris- 
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ing the steps of: receiving a multiplexed packet stream 
in which a plurality of object data are multiplexed, and 
separating packets containing N pieces of information 
relating to a program from the multiplexed packet 
stream with reference to packet identification numbers 5 
added to the respective packets; analyzing the N pieces 
of information relating to the program, and obtaining a 
packet identification number PID_syn of a packet con- 
taining object composition information, and a packet 
identification number PID_od of a packet containing 10 
information relating to the object data; and separating 
packets, the packet identification numbers of which 
match the packet identification numbers PID_syn and 
PID__od, from the multiplexed packet stream. 

According to a sixteenth aspect of the present 15 
invention, there is provided a data recording medium in 
which a program is recorded, and the program com- 
prises the steps of: receiving a multiplexed packet 
stream in which a plurality of object data are multi- 
plexed, and separating packets containing N pieces of 20 
information relating to a program from the multiplexed 
packet stream with reference to packet identification 
numbers added to the respective packets; analyzing the 
N pieces of information relating to the program, and 
obtaining a packet identification number PID_syn of a 25 
packet containing object composition information, and a 
packet identification number PID_od of a packet con- 
taining information relating to the object data; and sepa- 
rating packets, the packet identification numbers of 
which match the packet identification numbers PID_syn 30 
and PID_od, from the multiplexed packet stream. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a block diagram illustrating a demulti- 35 
plexing apparatus according to a first embodiment 
of the invention. 

Figure 2 is a diagram illustrating a data format of a 
multiplexed packet stream input to a demultiplexer 
according to the first embodiment. 40 
Figures 3(a)-3(d) illustrate data tables of packets in 
the multiplexed packet stream shown in figure 2 
and, more specifically, 3(a) shows a program asso- 
ciation table, 3(b) shows a program map table, 3(c) 
shows an initial object descriptor section, and 3(d) 45 
shows an object descriptor section. 
Figure 4 is a block diagram illustrating a demulti- 
plexing apparatus according to a second embodi- 
ment of the invention. 

Figures 5(a)-5(c) are diagrams for explaining a data so 
recording medium containing a program for imple- 
menting a demultiplexing process according to the 
first or second embodiment, using a computer sys- 
tem. 



DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

[Embodiment 1] 

Figure 1 is a block diagram for explaining an appa- 
ratus for demultiplexing multiplexed data (hereinafter 
referred to simply as "a demultiplexing apparatus") 
according to a first embodiment of the present inven- 
tion. 

As shown in figure 1 , the demultiplexing apparatus 
according to the first embodiment comprises a demulti- 
plexer 301 which receives a multiplexed packet stream 
in which a plurality of object data are multiplexed, and 
demultiplexes the multiplexed data with reference to 
packet identification numbers given to the packets; a 
first decoder 302 which decodes first object data sepa- 
rated from the multiplexed data and outputs first 
decoded object data; a second decoder 303 which 
decodes second object data separated from the multi- 
plexed data and outputs second decoded object data; a 
first memory 304 which stores a program association 
table (first information relating to a program) separated 
by the demultiplexer 301 ; a second memory 305 which 
stores a program map table (second information relating 
to the program) separated by the demultiplexer 301; a 
third memory 306 which stores an initial object descrip- 
tor section (third information relating to the program) 
separated by the demultiplexer 301; a fourth memory 
307 which stores an object composition information 
section separated by the demultiplexer 301 ; a fifth mem- 
ory 308 which stores an object descriptor section (infor- 
mation relating to the object data) separated by the 
demultiplexer 301 ; a compositor 309 which composites 
the first decoded object data and the second decoded 
object data; and an information analyzer 310 which ana- 
lyzes the packet identification numbers from the respec- 
tive data stored in the first to fifth memories 304, 305, 
306, 307, and 308. 

Figure 2 is a diagram showing a data format of the 
multiplexed packet stream input to the demultiplexer 
301 . Referring to figure 2, the multiplexed packet stream 
is divided into a plurality of fixed-length packets which 
are defined as TS (Transport Stream) packets in 
MPEG2, and each packet is given a packet identification 
number called PID (Packet Identifier). In a packet the 
PID of which is 0, a program association table (PAT) is 
stored. In a packet the PiD of which is 100, a program 
map table (PMT) is stored. In a packet the PID of which 
is 1 1 0, an initial object descriptor section (Initial OD sec- 
tion) is stored. In a packet the PID of which is 101, 
object data of MPEG2 video is stored. In a packet the 
PID of which is 102, object data of MPEG1 audio is 
stored. In a packet the PID of which is 103, an object 
composition information section is stored. In a packet 
the PID of which is 104, an object descriptor section 
(OD section) is stored. In a packet the PID of which is 
105, object data of MPEG4 video (#1) is stored. In a 
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packet the PID of which is 106, object data of MPEG4 
video (#2) is stored. In this first embodiment, the multi- 
plexed packet stream is demultiplexed with reference to 
the PIDs given to the respective packets, by analyzing 
the above-mentioned data. 

Figures 3(a)-3(d) are diagrams showing portions of 
the program association table (PAT), the program map 
table (PMT), the initial object descriptor section (Initial 
OD), and the object descriptor section (OD section), 
respectively. The program association table (PAT) 
shown in figure 3(a) has program numbers of the pro- 
gram map table, and identification numbers PMT_PID 
of packets corresponding to the respective program 
numbers. The program map table (PMT) shown in figure 
3(b) has stream types as data types, and identification 
numbers ES_PID of packets corresponding to the 
respective stream types. The data type is information 
indicating the kind of data content of the packet speci- 
fied by the PID. In figure 3(b), ES_PID_101 indicates 
that the data type is MPEG2 Video, ES_PID_102 indi- 
cates that the data type is MPEG1 Audio, and 
ES„PID_110 indicates that the data type is MPEG4. 
The initial object descriptor section (Initial OD section) 
shown in. figure 3(c) has stream types as data types, 
and identification numbers ES_PID of packets corre- 
sponding to' the respective stream types. In figure 3(c), - 
ES_PID_J03 indicates that the data type is object com- 
position information (SD), and ES_P!D_104 indicates 
that the data type is object descriptor section (OD sec- 
tion). The object descriptor section (OD section) shown 
in figure 3(d) has stream types as data types, and iden- 
tification numbers ESJPID of packets corresponding to 
the respective stream types. The object descriptor sec- 
tion further includes a stream map table (SMT) describ- 
ing ES_PID of MPEG4 data although it is not shown in 
figure 3(d). In figure 3(d), ES_PID_105 indicates that 
the data type is MPEG4 Video (#1), and ES„PID_106 
indicates that the data type is MPEG4 Video (#2). 

Hereinafter, the operation of the demultiplexing 
apparatus according to the first embodiment will be 
described using figures 1 , 2, and 3(a)-3(d). 

When the demultiplexer 301 receives the multi- 
plexed packet stream shown in figure 2, it separates the 
packet, the PID of which is 0, from the stream. This 
packet is output as a program association table and 
stored in the first memory 304. 

The information analyzer 310 receives the program 
association table from the first memory 304 and ana- 
lyzes this table. To be specific, the analyzer 310 
searches the table for a program number corresponding 
to a program to be received by the demultiplexing appa- 
ratus, and obtains PMT_PID_m which is a PID value of 
a program map table associated to the program 
number. Then, the information analyzer 310 outputs the 
PMT_PID_m to the demultiplexer 301. In this first 
embodiment, it is assumed that the analyzer 310 
obtains PMT_PID_100 shown in figure 3(a) as the 
PMT PID_m. 



Receiving the PMT_PID_m, the demultiplexer 301 
separates a packet, the PID of which matches the 
MPT_PID_m, from the multiplexed packet stream 
shown in figure 2 (in this case, a packet having 

5 PID_100). This packet is output as a program map table 
and stored in the second memory 305. 

The information analyzer 310 receives the program 
map table from the second memory 305, and analyzes 
this table to obtain the PID of packet of data related to 

io the program to be received by the demultiplexing appa- 
ratus (in this case, ES_PID_101 and ES_PID_102 
shown in figure 3(b)), and the identification number 
PID_iod of packet containing information for expressing 
the meaning in MPEG4 (ES_PID_110 shown in figure 

is 3(b)). Then, the information analyzer 310 outputs the 
ES_PID_1 10 as the PIDJod value of the packet relating 
to MPEG4. 

The demultiplexer 301 separates a packet, the PID 
of which matches the ES._PID.J1 10, from the multi- 

20 plexed packet stream shown in figure 2. This packet is 
output as an initial object descriptor section and stored 
in the third memory 306. 

The information analyzer 310 receives the initial 
object descriptor section from the third memory 306, 

25 and analyzes this table to obtain the identification 
number PID__sd indicating that the data type is the 
object composition information section (object composi- 
tion information data or scene description data) (in this 
case, ES_PID_103 shown in figure 3(c)), and the identi- 

30 fication number PID_od indicating that the data type is 
the object descriptor section (object descriptors) 
(ES_PID_104 shown in figure 3(c)). These PIDs are 
output to the demultiplexer 301 . 

The demultiplexer 310 separates the packet of the 

35 object composition information section indicated by the 
PID_sd and the packet of the object descriptor section 
indicated by the PID_od from the multiplexed packet 
stream shown in figure 2, and stores the. object compo- 
sition information section and the object descriptor sec- 

40 tion in the fourth memory 307 and the fifth memory 308, 
respectively. 

Next, the information analyzer 310 receives the 
object composition information from the object composi- 
tion information section stored in the fourth memory 

45 307, analyzes this information, and outputs composition 
information, such as composition position and composi- 
tion order, toward the compositor 309. Further, the infor- 
mation analyzer 310 receives the object descriptor 
section from the fifth memory 308, and analyzes this 

so section to obtain the data type indicating video or audio 
data coded by MPEG4 from the SMT included in this 
section. The analyzer 310 outputs this data type to the 
demultiplexer 301. In this first embodiment, there are 
two pieces of data having this data type, and these data 

55 are respectively described as first object data and sec- 
ond object data. For example, the PID value of the first 
object data is described PID_d1 while the PID value of 
the second object data is described as PID_d2. Refer- 
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ring to figure 3(d), it is assumed that the PID_d1 is 
ES_PID_105 r and the PID_d2 is ES_PID_106. 

The demultiplexer 301 separates the packets indi- 
cated by PID_d1 and PID_d2 from the multiplexed 
packet stream shown in figure 2, and outputs these 5 
packets to the first decoder 302 and the second 
decoder 303, respectively. To be specific, the first object 
data indicated by ES_PID_1 05 (PID_d1) is output to the 
first decoder 302 to be decoded, and the second object 
data indicated by ES_P1D_106 (PID_d2) is output to the 10 
second decoder 303 to be decoded. 

In the compositor 309, the first and second 
decoded object data respectively decoded by the first 
and second decoders 302 and 303 are composited 
according to the composition information, such as com- is 
position position and composition order, supplied from 
the information analyzer 310. 

The demultiplexing apparatus according to the first 
embodiment of the invention has various effects as fol- 
lows. 20 

Since the information analyzer 310 obtains the 
PIDs in the initial OD section in addition to the PIDs of 
the respective data described in the program map table, 
and outputs these PIDs toward the demultiplexer 301, 
reliable separation of MPEG4 data is realized. 25 

Further, since the PIDs of the packets for transmit- 
ting the object composition information and the object 
descriptors can be identified by the data types indicating 
the meaning in MPEG4 described in the initial object 
descriptor section, these PIDs can be identified with 30 
reliability. 

Moreover, since the IDs of the object descriptors 
described in the object composition information are 
compared with the IDs of the respective object descrip- . 
tors in the object descriptor section, the object composi- 35 
tion information can be associated with the object data 
designated by the object descriptor, whereby the object 
data separated by the demultiplexer 301 can be com- 
posited according to the composition position and the 
composition order described in the composition infor- 40 
mation. 

[Embodiment 2] 

Figure 4 is a block diagram for explaining a demulti- ' 45 
plexing apparatus according to a second embodiment 
of the present invention. 

As shown in figure 4, the demultiplexing apparatus 
according to the second embodiment comprises a 
demultiplexer 1 which receives a multiplexed packet so 
stream in which a plurality of object data are multi- 
plexed, and demultiplexes the multiplexed data with ref- 
erence to packet identification numbers given to the 
packets; a first decoder 2 which decodes first object 
data separated from the multiplexed data and outputs 55 
first decoded object data; a second decoder 3 which 
decodes second object data separated from the multi- 
plexed data and outputs second decoded object data; a 



first memory 4 which stores a program association table 
(first information relating to a program) separated by the 
demultiplexer 1 ; a second memory 5 which stores a pro- 
gram map table (second information relating to the pro- 
gram) separated by the demultiplexer 1 ; a third memory 
6 which stores object composition information of an 
object composition information section separated by the 
demultiplexer 1 ; a fourth memory 7 which stores object 
descriptors of an object descriptor section having infor- 
mation relating to the object data separated by the 
demultiplexer 1; a compositor 8 which composites the 
first decoded object data and the second decoded 
object data; and an information analyzer 9 which ana- 
lyzes the packet identification numbers from the first 
information relating to the program, the second informa- 
tion relating to the program, the object composition 
information, or the object descriptors. 

A description is now given of the operation of the 
demultiplexing apparatus according to the second 
embodiment. 

The data input to the demultiplexer 1 is a multi- 
plexed packet stream which is defined as a transport 
stream (TS) in MPEG2. Each packet in the multiplexed 
packet stream is given a packet identification number 
called PID (packet identifier). Initially, the demultiplexer 
1 separates a packet the PID of which is 0, and outputs 
this packet as a program association table to be stored 
in the first memory 4. 

The information analyzer 9 receives the program 
association table from the first memory 4, searches the 
table for a program number corresponding to a program 
to be received by the demultiplexing apparatus, and 
obtains PID_m which is a value of a program map PID 
associated to the program number. Then, the analyzer 9 
outputs the PID_m to the demultiplexer 1 . 

Receiving the PID_m, the demultiplexer 1 sepa- 
rates a packet, the PID of which matches the program 
map PI Dm, from the multiplexed packet stream, and 
outputs this packet as a program table to be stored in 
the second memory 5. 

The information analyzer 9 receives the program 
map table from the second memory 5. and obtains the 
PID of packet of data related to the program to be 
received by the demultiplexing apparatus. 

Further, the information analyzer 9 analyzes infor- 
mation for expressing the meaning in MPEG4, which 
information is described in relation to the PID of each 
data in the program map table. To be specific, this infor- 
mation comprises the data types in MPEG4, the object 
IDs of the MPEG4 objects described by the data types, 
and the PIDs of the MPEG4 packets. The data type indi- 
cates the kind of data content of the packet specified by 
the PID. There are three kinds of data contents as fol- 
lows: an object composition information section, an 
object descriptor section, and data other than these 
sections. The information analyzer 9 analyzes the data 
type, and outputs the identification number PID_syn of 
the packet including the object composition information 
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(object composition information section) and the identi- 
fication number PID_od of the packet including the 
object descriptors (object descriptor- section), toward 
the demultiplexer 1 . 

When the data type detected by the information 
analyzer 9 from the program map table is neither the 
object composition information section nor the object 
descriptor section, the information analyzer 9 obtains 
the object ID. When an MPEG4 FlexMux channel (FMC) 
is described, the information analyzer 9 obtains this as 
well. The FMC means a packet identification number in 
MPEG4 packetization. In this second embodiment, 
there are two pieces of data having the data type other 
than the above-mentioned sections, and these data are 
respectively described as the first object data and the 
second object data. In the first object data, no FMC is 
used. Its PID value is PID_d1 , and its object ID value is 
ODJdjdl. In the second object data, FMC_d2 is 
described as an FMC value. Its PID value is PID_d2, 
and its object ID value is OD_id_d2. The information 
analyzer 9 outputs PID_d1, PID_d2, and FMC_d2 
toward the demultiplexer 1 . 

Receiving PID_syn, PID_od, PID_d1, PlD_d2, and 
FMC_d2, the demultiplexer 1 separates a packet the 
PID of which is PID_syn (object composition information 
section) from the multiplexed packet stream, and out- 
puts this packet as object composition information to the 
third memory 6. Further, the demultiplexer 1 separates 
a packet the PID of which is PID_od (object descriptor 
section) from the multiplexed packet stream, and out- 
puts this packet as object descriptors to the fourth mem- 
ory 7. Further, the demultiplexer 1 separates a packet, 
the PID of which is PID_d1, and outputs this packet to 
the first decoder 2. Furthermore, the demultiplexer 1 
separates a packet, the PID of which is PID_jd2 and the 
FMC of which is FMC_d2, and outputs this packet to the 
second decoder 3. 

Receiving the object composition information, the 
information analyzer 9 obtains information such as the 
composition position of objects and the hierarchy of 
objects, and the object IDs of the respective objects, 
and outputs these to the compositor 8. Further, the 
information analyzer 9 receives an object descriptor to 
which the object ID value of the first object data, 
OD_id_d1, is given, and obtains a coding scheme 
Code_d1, a code AL_predefined__d1 indicating whether 
the MPEG4 packetization using the Access Unit Layer 
exists or not, and a code AL_config_d1 indicating set- 1 
ting information of the packetization. The information 
analyzer 9 analyzes these codes to create access-unit- 
layer instruction information AL_d1, and outputs AL_d1 
and Code_d1 to the first decoder 2. Moreover, the infor- 
mation analyzer 9 receives an object descriptor to which 
the object ID value of the second object data, 
OD_id_d2, is given, and obtains a coding scheme 
Code_d2, a code AL_predef ined__d2 indicating whether 
the MPEG4 packetization using the Access Unit Layer 
exists or not, and a code AL_config_d2 indicating set- 



ting information of the packetization. The information 
analyzer 9 analyzes these codes to create access-unit- 
layer instruction information AL_d2, and outputs AL_d2 
and Code_d2 to the second decoder 3. 

5 The first decoder 2 decodes the first object data 

according to the coding scheme Code_d1, and outputs 
the decoded data. The second decoder 3 decodes the 
second object data according to the coding scheme 
Code_d2, and outputs the decoded data. 

10 The compositor 8 receives the decoded first object 
data and the decoded second object data, and compos- 
ites the first and second object data with reference to 
the object IDs and the information relating to the object 
hierarchy and the like, which are supplied from the infor- 

15 mation analyzer 9, and outputs the composite data. 

The above-mentioned demultiplexing apparatus 
according to the second embodiment provides the fol- 
lowing effects. 

Since the information analyzer 9 obtains the PID of 

20 the MPEG4 multiplexed packet in addition to the PID of 
each data described in the program map table, and out- 
puts this PID to the demultiplexer 1, reliable separation 
of the MPEG4 data is realized. 

Further, since the P IDs of the packets for transmit- 

25 ting the object composition information section and the 
object descriptor section are identified by the data types 
described in the program map table and indicating the 
meaning in MPEG4, these PIDs can be identified with 
reliability. 

30 Furthermore, since the object IDs in MPEG4 are 
obtained with respect to the PIDs of the respective data 
described in the program map table, and these object 
IDs are compared with the object IDs described in the 
object descriptor section, the object IDs can be associ- 

35 ated with the PIDs, whereby the respective data sepa- 
rated with PIDs can be composited using the object IDs 
described in the object composition information section. 

Moreover, when the information analyzer 9 receives 
the object descriptors, it analyzes the code indicating 

40 whether the MPEG4 Access Unit Layer is used or not 
and the code indicating the setting information, and out- 
puts the result of the analysis together with the informa- 
tion indicating the coding scheme toward the decoder. 
So, the decoder can perform appropriate header 

45 processing of the Access Unit Layer and appropriate 
decoding, and this enables packetization using no 
MPEG4 Access Unit Layer. Accordingly, a broadcasting 
in which MPEG4 object data is mixed can be received 
by the conventional MPEG2 receiver, while PES-pack- 

50 etized data can be received by the MPEG4 receiver. 

In the first (second) embodiment described above, 
when the information analyzer 310 (9) receives the 
object composition information and the object descrip- 
tors to analyze the contents thereof, it may receive 

55 these data to which version numbers, the values of 
which vary when the contents of these data vary, are 
added as header information. In this case, the informa- 
tion analyzer 310 (9) stores the version numbers of 
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these data once analyzed and, when these data are 
input nest, the version numbers of the input data are 
compared with the version numbers stored in the ana- 
lyzer 310 (9). Only when the version numbers do not 
match, the analyzer 310 (9) analyzes the data again. 5 
Thereby, the burden on data analysis by the information 
analyzer 310 (9) is significantly reduced. 

Further, the information analyzer 310 (9) may 
receive the data obtained by dividing the object compo- 
sition information and the object descriptors into a plu- 10 
rality of sections and giving a section number at the 
head of each section. In this case, the information ana- 
lyzer 310 (9) can restore the object composition infor- 
mation and the object descriptors by integrating the 
sections. is 

Furthermore, the information analyzer 310 (9) may 
receive the data to which an error detection code is 
added at the end of each section. In this case, the infor- 
mation analyzer 310 (9) can check whether the data 
within each section includes a data error or not, by refer- 20 
ring to the error detection code. As an error detection 
code, a CRC (Cyclic Redundancy Check) code or a 
checksum is available. 

While in the first (second) embodiment of the inven- 
tion the demultiplexer 301 (1), the first and second 25 
decoders 302 and 303 (2 and 3), the compositor 309 
(8), and the information analyzer 310 (9) are described 
as individual units, the present invention is not restricted 
thereto. In an actual hardware structure, these units can 
be implemented by at least one LSI or microprocessor. 30 
Further, it is needless to say that the first to fifth memo- 
ries can be implemented by at least one memory, such 
as a DRAM (Dynamic Random Access Memory) or a 
SDRAM (Synchronous Dynamic Random Access Mem- 
ory). 35 

While in the first and second embodiments the 
demultiplexers 301 and 1 receive a multiplexed packet 
stream in which packets are multiplexed by MPEG2, 
these demultiplexers may receive multiplexed packet 
streams obtained by other schemes, such as a multi- 40 
plexing scheme defined in ITU-TH.223, or TCP (Trans- 
mission Control Protocol), UDP (User Datagram 
Protocol), and RTP (Realtime Transport Protocol), 
which protocols are used in internet. 

Moreover, while the demultiplexing apparatuses 45 
according to the first and second embodiments are 
implemented by hardware, the present invention is not 
restricted thereto. The demultiplexer, the first and sec- 
ond decoders, the compositor, and the information ana- 
lyzer according to the aforementioned embodiments so 
may be implemented as a software program in which 
the functions of these units are programmed, with the 
same effects as described for the aforementioned 
embodiments. 

55 

[Embodiment 3] 

Figures 5(a)-5(c) are diagrams for explaining the 



case where the demultiplexing process according to the 
first or second embodiment is implemented by a compu- 
ter system using a floppy disk containing a program of 
the demultiplexing process. More specifically, figure 5(b) 
shows a front view of a case F containing the floppy 
disk, a cross-sectional view thereof, and the floppy disk 
FD (recording medium body). Figure 5(a) shows an 
example of a physical format of the f loppy disk FD. At 
the surface of the disk, a plurality of tracks Tr are formed 
concentrically from the outer circumference of the disk 
toward the inner circumference. Each track is divided 
into 16 sectors in the angular direction. Therefore, in the 
floppy disk containing the above-mentioned program, 
data of the program are recorded in assigned sectors on 
the floppy disk FD. 

Figure 5(c) shows the structure for recording/repro- 
ducing the program in/from the floppy disk FD. When 
the program is recorded in the floppy disk FD, data of 
the program are written in the floppy disk FD from the 
computer system Cs through the floppy disk drive FDD. 
When the above-mentioned demultiplexing apparatus is 
constructed in the computer system Cs from the pro- 
gram recorded in the floppy disk FD, the program is read 
from the floppy disk FD by the floppy disk drive FDD and 
then loaded to the computer system Cs. 

As described above, when a program for imple- 
menting the present invention by software is recorded in 
a data recording medium such as a floppy disk, demul- 
tiplexing of multiplexed data according to the aforemen- 
tioned embodiments can be easily executed in an 
independent general-purpose computer system. 

While in this third embodiment a floppy disk is 
employed as a data recording medium, an optical disk 
may be employed in similar way. Further, applicable 
recording media are not restricted thereto, namely, any 
recording media, such as an IC card and a ROM cas- 
sette, may be employed as long as the above-men- 
tioned program can be recorded therein. 

Claims 

1 . A demultiplexing apparatus comprising: 

demultiplexing means for receiving a multi- 
plexed packet stream in which a plurality of 
object data are multiplexed, and demultiplexing 
the multiplexed data with reference to packet 
identification numbers added to the respective 
packets; 

decoding means for decoding M pieces of 
object data separated from the multiplexed 
data, and outputting M pieces of decoded 
object data; 

a memory for storing N pieces of information 
relating to a program and separated by the 
demultiplexing means; 

a memory for storing object composition infor- 
mation separated by the demultiplexing means; 



DOC1D: <EP . 0B93926A2 I > 



15 



EP 0 893 926 A2 



16 



3. 



a memory for storing information relating to the 
object data and separated by the demultiplex- 
ing means; 

composition means for compositing the M 
pieces of decoded object data; and s 
information analysis means for analyzing the 
packet identification numbers from the N 
pieces of information relating to the program, 
the object composition information, or the infor- 
mation relating to the object data. io 

A demultiplexing apparatus as defined in Claim 1 
wherein: 

said information analysis means analyzes the 15 
N pieces of information relating to the program, 
obtains a packet identification number PID_syn 
of a packet containing the object composition 
information, and outputs the packet identifica- 
tion number PID_syn to the demultiplexing .20 
means; and 

said demultiplexing means separates the 
packet containing the object composition infor- 
mation from the input multiplexed packet 
stream with reference to the packet identif ica- 25 
tion number PID_syn, and outputs this packet. 

A demultiplexing apparatus as defined in Claim 1 
wherein: 

said information analysis means analyzes the 
N pieces of information relating to the program, 
obtains a packet identification number PID_od 
of a packet containing the information relating 
to the object data, and outputs the packet iden- 
tification number PID_od to the demultiplexing 
means; and 

said demultiplexing means separates the 
packet containing the information relating to 
the object data from the input multiplexed 40 
packet stream with reference to the packet 
identification number PID_od, and outputs this 
packet. 

A demultiplexing apparatus as defined in Claim 1 45 8. 
wherein: 



30 



35 



said information analysis means analyzes the 
information relating to the object data, judges a 
code Al_predef ined for each of the object data, 
which code indicates whether packetization 
according to the international standard MPEG4 
exists or not, and outputs the code 
Al__predefined toward the demultiplexing 
means; and 

said demultiplexing means performs separa- 
tion of the MPEG4 packetization for each 
object data with reference to the code 
Al_predefined. 

A demultiplexing apparatus as defined in Claim 1 
wherein: 

said information analysis means analyzes the 
information relating to the object data, judges a 
code Al _predef ined for each of the object data, 
which code indicates whether packetization 
according to the international standard MPEG4 
exists or not, and outputs the code 
Al_predefined toward the decoding means; 
and 

said decoding means performs separation of 
the MPEG4 packetization with reference to the 
code Al_predef ined. 

A demultiplexing apparatuses defined in Claim 1 
wherein: 

said information analysis means analyzes the 
information relating to the object data, judges a 
code ALconfig for each of the object data, 
which code indicates setting information of a 
packet header according to the international 
standard MPEG4, and outputs the code 
AI__config toward the demultiplexing means; 
and 

said demultiplexing means performs separa- 
tion of the MPEG4 packetization for each, 
object data with reference to the code 
A!_config. 

A demultiplexing apparatus as defined in Claim 1 
wherein: 



5. 



said information analysis means analyzes the 
object composition information, and outputs 
information indicating the hierarchical relation- 
ship of the object data toward the composition 
means; and 

said composition means composites the M 
pieces of decoded object data with reference to 
the hierarchical relationship. 

A demultiplexing apparatus as defined in Claim 1 
wherein: 



50 



said information analysis means analyzes the 
information relating to the object data, judges a 
code AI_config for each of the object data, 
which code indicates setting information of a 
packet header according to the international 
standard MPEG4, and outputs the code 
AI_config toward the decoding means; and 
said decoding means performs separation of 
the MPEG4 packetization with reference to the 
code ALconf ig. 
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9. A demultiplexing apparatus as defined in Claim 1 
wherein: 

said information analysis means analyzes the 
N pieces of information relating to the program, s 
obtains a packet identification number PID_d of 
a packet containing the object data and a 
packet identification number FMCjd according 
to the international standard MPEG4, and out- 
puts these identification numbers toward the 10 
demultiplexing means; and 
said demultiplexing means separates the 
packet containing the object data from the input 
multiplexed packet stream with reference to the 
identification numbers PID_d and FMC_d, and 15 
outputs the packet. 

10. A demultiplexing apparatus as defined in Claim 1 
wherein said information analysis means obtains a 
code for deciding whether the packet identification 20 
number FMC_d according to the international 
standard MPEG4 is added or not, by analyzing the 

N pieces of information relating to the program. 

11. A demultiplexing apparatus as defined in Claim 1 25 
wherein said information analysis means receives 
the object composition information which is divided 
into at least one section, and integrates these sec- 
tions with reference to version numbers and section 
numbers which are added at the heads of the 30 
respective sections, thereby obtaining the object 
composition information. 

12. A demultiplexing apparatus as defined in Claim 1 
wherein said information analysis means receives 35 
the information relating to the object data, which 
information is divided into at least one section, and 
integrates these sections with reference to version 
numbers and section numbers which are added at 

the heads of the respective sections, thereby 40 
obtaining the information relating to the object data. 

13. A demultiplexing apparatus as defined in Claim 1 
wherein said information analysis means receives 
the object composition information which is divided 45 
into at least one section, decides whether data of 
each section has a data error or not according to an 
error detection code added to the end of each sec- 
tion, and discards the section having a data error. 

50 

14. A demultiplexing apparatus as defined in Claim 1 
wherein said information analysis means receives 
the information relating to the object data, which is 
divided into at least one section, decides whether 
data of each section has a data error or not accord- 55 
ing to an error detection code added to the end of 
each section, and discards the section having a 
data error. 



15. A demultiplexing method comprising the steps of: 

receiving a multiplexed packet stream in which 
a plurality of object data are multiplexed, and 
separating packets containing N pieces of 
information relating to a program from the mul- 
tiplexed packet stream with reference to packet 
identification numbers added to the respective 
packets; 

analyzing the N pieces of information relating 
to the program, and obtaining a packet identifi- 
cation number PID_syn of a packet containing 
object composition information, and a packet 
identification number PID__od of a packet con- 
taining information relating to the object data; 
and 

separating packets, the packet identification 
numbers of which match the packet identifica- 
tion numbers PID_syn and PID_od, from the 
multiplexed packet stream. 

16. A data recording medium in which a program is 
recorded, said program comprising the steps of: 

receiving a multiplexed packet stream in which 
a plurality of object data are multiplexed, and 
separating packets containing N pieces of 
information, relating to a program from the mul- 
tiplexed packet stream with reference to packet 
identification numbers added to the respective 
packets; 

analyzing the N pieces of information relating 
to the program, and obtaining a packet identifi- 
cation number PID_syn of a packet containing 
object composition information, and a packet 
identification number PID_od of a packet con- 
taining information relating to the object data; 
and 

separating packets, the packet identification 
numbers of which match the packet identifica- 
tion numbers PID_syn and PID_od, from the 
multiplexed packet stream. 
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Fig. 3 (a) program association table (PAT) 



program number 


PMT_PID 


1 


100 


2 


200 


3 


300 







Fig. 3 (b) program map table (PMT) 
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Fig .3 (c) initial object descriptor section (Initial OD section) 



stream type 


ES_PID 


SD Stream 
OD Stream 


103 
104 



Fig .3 (d) object descriptor section (OD section) 
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Fig.5 (a) 
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(54) Apparatus and method for demultiplexing multiplexed data 



(57) Disclosed is an apparatus for demultiplexing 
multiplexed data. This apparatus comprises a demulti- 
plexer for receiving a multiplexed packet stream in 
which a plurality of object data are multiplexed, and 
demultiplexing the multiplexed data with reference to 
packet identification numbers added to the respective 
packets; a decoder for decoding M pieces of object data 
separated from the multiplexed data, and outputting M 
pieces of decoded object data; a memory for storing N 
pieces of information relating to a program and sepa- 
rated by the demultiplexer; a memory for storing object 
composition information separated by the demulti- 
plexer; a memory for storing information relating to the 
object data and separated by the demultiplexer; a com- 
positor for compositing the M pieces of decoded object 
data; and an information analyzer for analyzing the 
packet identification numbers from the N pieces of infor- 
mation relating to the program, the object composition 
information, or the information relating to the object 
data. Therefore, when a multiplexed packet stream in 
which MPEG4 data are multiplexed by MPEG2 is 
demultiplexed, the MPEG4 data can be separated arid 
decoded with high reliablity. 
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